The landmark Transformer [9] model has demonstrated impressive performance across a wide range of scenarios, extending well beyond the realm of Natural Language Processing. The potential of multi-head self-attention method lies in the ability to pick up a signal from any data modality, provided it exhibits a spatial (e.g., sequential) structure and is appropriately preprocessed into discrete tokens for model consumption. However, in practice, the convergence rate of Transformer models in default configurations is considered unsatisfactory. This issue can be mitigated by incorporating prior domain knowledge and inductive biases during the tokenization phase, making the data more suitable for processing by the algorithm. In the field of Computer Vision, the Hybrid Vision Transformers approach [2] has shown that leveraging a pre-trained convolutional backbone as a feature extractor leads to faster convergence and improved downstream performance. Similar observations have been made in customer modeling for personalization [7], where purchase and non-purchase actions were pre-embedded before processing with a BERT-like model. In the healthcare sector, a sequence of electronic health records was preprocessed based on domain expert knowledge to be further consumed by a Transformerbased model with an objective to predict the next medical code [5]. Inspired by these advances, we propose a method for modeling in-game player behavior data that employs a structured approach to convert tracking events into dense tokens. We benchmark and compare the proposed approach against the tabular and semi-structured baselines.